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Abstract: 

The purpose of this research is to adapt The Teacher Effectiveness Scale in Higher 
Education into Turkish language. The survey method was employed. Collected data 
were analyzed using confirmatory and exploratory factor analyses. The adapted scale 
had 32 items and 4 factors explaining 50.3% variance. Cronbach alpha reliability 
coefficients of the factors varied from a=0.70 to a=0.89 and factor loadings of the items 
ranged from 0.47 to 0.80. Statistically significant correlations among factors ranging 
from r=0.54 to r=0.58 were found. It is concluded that the adapted scale is a valid and 
reliable instrument to measure teaching-related behavior, subject matter expertise, 


relational expertise, and personality aspects of teacher effectiveness in higher education. 
Keywords: higher education, teacher effectiveness, scale, adaptation 
1. Introduction 


Every human activity needs to be evaluated for its effectivity. In a broad view, 
evaluation means any systematic examination of employee’s performance (Mercer, 
Barker, & Bird, 2010, p. 139). It is generally thought as the last step of management 
process and includes the utilization of data for improvement and correction (Basar, 
2000, p. 55). Its aim is to determine the success level of the performance objectively 


(Bursahioglu, 2011, s. 125). Evaluation has an important potential as a data source which 
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informs the organizational system supporting the teaching and meaningful feedback for 
improving teaching practices (Maslow & Kelley, 2012). 

As a major component of the education, teacher evaluation is at the core. 
Gathering teacher evaluation is undeniably helpful in identifying exemplary teacher 
and teaching in higher education (Feldman, 2007, p. 118). It is also helpful for on-going 
self-monitoring of one’s teaching, evaluating one’s professional development needs, 
and preparing a case for promotion or tenure, providing information for students to use 
in the selection of courses and instructors, and providing an outcome for research 
(Casey, Gentile, & Bigger, 1997; Marsh, 1984). There are generally two fundamental 
aspects of teacher evaluation which include improvement function which relates to 
formative nature and accountability function which relates to summative nature 
(Tucker & Stronge, 2005, p. 6-7). Classroom observations, principal evaluations, analysis 
of classroom artifacts, portfolios, self-reports of teacher practice, and value-added 
models are the methods of evaluating teacher effectiveness (Goe, Bell, & Little, 2008). In 
addition, one of the key components of teacher evaluation can be thought as student 
evaluation of teachers as a recipient of service provided and affected. Implementing 
student questionnaires which include different dimensions of teaching to evaluate 
teaching effectiveness and quality is a fairly common procedure and it aims the 
improvement of teaching quality (Dresel & Rindermann, 2011). They can provide 
reliable and valid information on the quality of higher education (Murray, 1983). 
Likewise, student questionnaires fulfilled anonymously is said to be a useful apparatus 
for performance evaluation of teachers (Marsall, 2012). They are also important because 
they cause the teaching staff to be politer towards students, to pay attention to class 
schedule especially for the beginning and end of the lectures, paying attention to 
assessment of students (Ergiin, 2001). 

Timing of the evaluation, anonymity of student raters, instructor presence in 
classroom, stated purpose of the evaluation might affect the process of student 
evaluation in higher education (Wachtel, 1998). Considering the topics for the teacher 
evaluation, following issues may be of importance (Feldman, 2007, p. 104-105); teacher’s 
preparation; organization of the course, clarity, teacher pursued and/or met course 
objectives, perceived outcome or impact of instruction, teacher’s stimulation of interest 
in the course and its subject matter, teacher motivates students to do their best; high 
standard of performance required, teacher’s encouragement of questions and openness 
to opinions of others, teacher’s availability and helpfulness, teacher’s elocutionary 
skills, clarity of course objectives and requirements, teacher’s knowledge of the subject, 
teacher’s sensitivity to and concern with class level and progress, teacher’s enthusiasm 


(for subject or for teaching), teacher’s fairness; impartiality of evaluation of students; 
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quality of examinations, classroom management, intellectual challenge and 
encouragement of independent thought (by the teacher and the course), personality 
characteristics, teacher’s concern and respect for students friendliness of the teacher, 
nature, quality, and frequency of feedback from the teacher to the students, 
pleasantness of classroom atmosphere, nature and value of the course (including its 
usefulness and relevance), difficulty of the course description, difficulty of the course 
evaluation, nature and usefulness of supplementary materials and teaching aids. 

There are many studies aiming the process of evaluation teacher effectiveness. 
Patrick and Smart (1998) developed a measure for evaluating teacher effectiveness. 
Following undergraduate students’ identification of qualities, they formed a meta 
inventory and it was revealed that respect for students, ability to challenge students, 
organization and presentation skills were three important factors for teacher evaluation. 
Karkoulian (2002) developed an appraisal practice for the Lebanese American 
University. As a result, a communication model for the performance appraisal scheme 
was formed. The researcher finally recommended the formal adoption of a performance 
appraisal process at Lebanese American University. Melnic, (2011) studied the 
evaluation of academics at George Bacovia University in Bacau. This study focused on 
the formative evaluation of courses, seminars, practical work; formative evaluation of 
research projects; evaluation from colleagues and experts; self-evaluation; and 
evaluation of management. As for the evaluation of the performance of the academics 
catching the students’ attention, introducing the subject, explaining the subject, the aids 
necessary for teaching and learning, keeping the students’ interest, teacher-student 
interaction, organization of students, retroaction, communication with students, the use 
of time; lecture summary constituted the evaluation. 

This research aims to adapt a teacher evaluation instrument into Turkish 
language. Considering that there is no instrument developed for the higher education, 


this effort may contribute to the area. 
2. Material and Methods 


The survey method was employed in this research. This method emphasizes collecting 
data from a large sample to produce generalizable results (Fraenkel, Wallen & Hyun, 
2012). 


2.1. Sample 
Data were collected from students attending at Suleyman Demirel University Foreign 


Language Preparation Class in Isparta, Turkey during the 2015-2016 academic year. All 


European Journal of Education Studies - Volume 3 | Issue 10 | 2017 286 


Thsan Marulcu, Kivane Bozkus 
ADAPTATION OF THE TEACHER EFFECTIVENESS SCALE IN 
HIGHER EDUCATION INTO TURKISH LANGUAGE 


452 students were reached without a sampling procedure. A total of 247 students 


volunteered for the research (Table 1). 


Table 1: Participants of the research 


Major Population Participants Rate of Return (%) 
Tourism Management 101 52 51.49 
Business Management 151 82 54.30 
Landscape Architecture 38 26 68.42 
Architecture 49 42 85.71 
City and Region Planning 39 17 43.59 
Electricity and Electronic Engineering 74. 28 37.84 
Total 452 247 54.65 


2.2. Instrument 
The Teacher Effectiveness Scale in Higher Education developed by Calaguas (2013) was 
used as the data collection instrument. The scale consists of 67 items under 4 factors. 


Cronbach alpha reliability coefficients of the factors varies from a=0.71 to a=0.97. 


2.3. Process 

A team of foreign language specialists has been formed and the items were translated 
into Turkish. Then the items were back-translated into English and compared to the 
original ones. The items that were not compatible with the original statements had the 


same process again with more elaboration. 


2.4. Analysis 

Confirmatory and exploratory factor analysis were applied to data collected from the 
students. To determine whether the original factor structure of the scale had been 
preserved after the adaptation process confirmatory factor analysis was conducted. 
After determining that the original factor structure had not been preserved, exploratory 


factor analysis to reveal the new factor structure was employed. 

3. Results 

Confirmatory factor analysis results presented in Table 2 revealed that many of the fit 
indexes were out of the desired range (Cokluk, Sekercioglu & Bityiikozttirk, 2014; Hu & 


Bentler, 1999). It has been concluded that the original factor structure of the scale had 
been changed. 
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Table 2: Confirmatory factor analysis results 


Fit Indexes Excellent Fit Criteria* Acceptable Fit Criteria* Actual Values Result 
X2/df(CMIN/DF) O0< X2/df<2 2< X2/df<3 1.88 Excellent Fit 
GFI .95 < GFI < 1.00 .80 < GFI < 95 0.63 No Fit 
CFI .95 < CFI < 1.00 90 < CFI < .95 0.72 No Fit 
NNFI 95 < TLI < 1.00 90 < TLI<s .95 0.55 No Fit 

IFI .95 < IFI < 1.00 90 <IFI < .95 0.72 No Fit 
RMSEA 00 < RMSEA < .05 05s RMSEA <.08 0.06 Acceptable Fit 
SRMR 00 < SRMR < .05 .05 < SRMR < .10 0.06 Acceptable Fit 


* According to Cokluk, Sekercioglu & Biytikéztiirk, 2014; Hu & Bentler, 1999. 


Kaiser-Meyer-Olkin adequacy of sample size indicator was calculated as 0.93 and the 
Bartlett’s test of sphericity was found as significant (p<0.001) for the exploratory factor 
analysis. The 35 items that have factor loadings and item-total correlations lower than 
0.40 were eliminated. Principal component analysis with Varimax rotation method 
revealed that the adapted scale has 32 items and 4 factors explaining 50.3% variance 
(Table 3). Cronbach alpha reliability coefficients of the factors varies from a=0.70 to 
a=0.89. Factor loadings of the items ranges from 0.47 to 0.80. 


Table 3: Comparison of the factor structures of the original and adapted scales 


Original Scale Adapted Scale 
if Factor Aas Variance fe Factor ree Variance 
actors ems a ems a 
Loadings Le (%) Loadings a (%) 
Teaching- 
; 45 51-.63 97 19 14 47-.65 .88 33.4 
Related Behavior 
Subject Matter 
j 10 51-.67 89 8.2 6 .53-.76 88 6.1 
Expertise 
Relational 
; 7 53-.63 83 7.8 7 .63-.80 89 7 
Expertise 
3.8 
Personality 5 52-.57 71 6.3 5 .48-.63 .70 
Total 67 51-.67 97 41.3 32 .47-.80 93 50.3 


Correlations among factors of the adapted scale were also analyzed. Statistically 
significant (p<0.001) correlations ranging from r=0.54 to r=0.58 among factors can be 


seen in Table 4. 
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Table 4: Correlations among factors 


Teaching-Related Subject Matter Relational : 
; : : Personality 
Behavior Expertise Expertise 
Subject Matter Expertise 4" 
Relational Expertise nay 56° 
Personality 56" 56" 58° 
Total 80° 81 84" 82° 


*p<0.001 
4. Conclusion 


The adapted scale is a valid and reliable instrument to measure multiple aspects of 
teacher effectiveness in higher education. It can be used to evaluate teachers in higher 
education for their teaching-related behavior, subject matter expertise, relational 
expertise, and personality. Given the fact that the adapted scale has 32 items, it can be 
considered as a concise instrument that is easy to apply. The adapted scale can be used 
both during and at the end of academic terms for formative and summative evaluation 
purposes. This way, it is assumed that teachers in higher education will have the 
opportunity to learn how the students perceive their effectiveness and to adjust their 
practices according to these perceptions. However, this assumption should be 


investigated via further research. 


References 


Basar, H. (2000). Egitim denetcisi (5th edition). Ankara, Turkey: Pegem. 

2. Bursalioglu, Z. (2011). Okul yoénetiminde yeni yapi ve davranis (16th edition). 
Ankara, Turkey: Pegem. 

3. Calaguas, G. M. (2013). Teacher effectiveness scale in higher education: 
Development and psychometric properties. International Journal of Research 
Studies in Education, 2(2), 3-20. 

4. Casey, R. J., Gentile, P., & Bigger, S. W. (1997). Teaching appraisal in higher 
education: An Australian perspective. Higher Education, 34(4), 459-482. 

5. Cokluk, O., Sekercioglu, G. & Buytikoztirk, $. (2014). Sosyal bilimler igin ¢ok 
degiskenli istatistik: SPSS ve LISREL uygulamalani (3rd edition). Ankara: Pegem. 

6. Dresel, M., & Rindermann, H. (2011). Counseling university instructors based on 


student evaluations of their teaching effectiveness: A multilevel test of its 


European Journal of Education Studies - Volume 3 | Issue 10 | 2017 289 


Thsan Marulcu, Kivanc Bozkus 
ADAPTATION OF THE TEACHER EFFECTIVENESS SCALE IN 
HIGHER EDUCATION INTO TURKISH LANGUAGE 


effectiveness under consideration of bias and unfairness variables. Research in 
Higher Education, 52(7), 717-737. 

7. Ergiin, M. (2001). Universitelerde 6&retim etkililiginin gelistirilmesi. 2000 Yilinda 
Turk Milli Egitim Orgiitii ve YOnetimi Ulusal Sempozyumu (p. 188-192). Ankara: 
Ogretmen Htiseyin Htisnti Tekisik Egitim Arastirma Gelistirme Vakfi. 

8. Feldman, K. (2007). Identifying exemplary teachers and teaching: Evidence from 
student ratings. In R. P. Perry, & J. C. Smart, The scholarship of teaching and 
learning in higher education: an evidence-based perspective (p. 93-130). Dordrecht, The 
Netherlands: Springer. 

9. Fraenkel, J. R., Wallen, N., & Hyun, H. (2012). How to design and evaluate research 
in education (8th ed.). New York: McGraw-Hill. 

10. Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A 
research synthesis. Washington: National Comprehension Center for Teacher 
Quality. 

11. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance 
structure analysis: Conventional criteria versus new alternatives. Structural 
equation modeling: a multidisciplinary journal, 6(1), 1-55. 

12. Karkoulian, 5S. K. (2002). Performance appraisal in higher education. UK: University 
of Leicester. 

13. Marsall, K. (2012). Fine-tuning teacher evaluation. Educational Leadership, 70(3), 
50-53. 

14.Marsh, H. W. (1984). Students’ evaluations of university teaching: 
Dimensionality, reliability, validity, potential biases, and utility. Journal of 
Educational Psychology, 76(5), 707-754. 

15. Maslow, v., & Kelley, C. (2012). Does evaluation advance teaching practice? The 
effects of performance evaluation on teaching quality and system change in large 
diverse high schools. Journal of School Leadership, 22(3), 600-632. 

16. Melnic, A. S. (2011). Evaluation in education - A formative approach in 
measuring the performances of academics. Economy Transdisciplinarity Cognition, 
14(1), 168-175. 

17. Mercer, J., Barker, B., & Bird, R. (2010). Human resource management in education 
contexts, themes and impact. New York, USA: Routledge. 

18. Murray, H. G. (1983). Low-inference classroom teaching behaviors and student 
ratings of college teaching effectiveness. Journal of Educational Psychology, 75(1), 
138-149. 


European Journal of Education Studies - Volume 3 | Issue 10 | 2017 290 


Thsan Marulcu, Kivanc Bozkus 
ADAPTATION OF THE TEACHER EFFECTIVENESS SCALE IN 
HIGHER EDUCATION INTO TURKISH LANGUAGE 


19. Patrick, J. & Smart, R. M. (1998). An empirical evaluation of teacher 
effectiveness: The emergence of three critical factors. Assessment & Evaluation in 
Higher Education, 23(2) 165-178. 

20. Tucker, P. D., & Stronge, J. H. (2005). Linking teacher evaluation and student 
learning. Danvers, USA: Association for Supervision and Curriculum 
Development. 

21. Wachtel, H. K. (1998). Student evaluation of college teaching effectiveness: A 
brief review. Assessment & Evaluation in Higher Education, 23(2) 191-211. 


European Journal of Education Studies - Volume 3 | Issue 10 | 2017 291 


Thsan Marulcu, Kivanc Bozkus 
ADAPTATION OF THE TEACHER EFFECTIVENESS SCALE IN 
HIGHER EDUCATION INTO TURKISH LANGUAGE 


Creative Commons licensing terms 

Author(s) will retain the copyright of their published articles agreeing that a Creative Commons Attribution 4.0 International License (CC BY 4.0) terms 
will be applied to their work. Under the terms of this license, no permission is required from the author(s) or publisher for members of the community 
to copy, distribute, transmit or adapt the article content, providing a proper, prominent and unambiguous attribution to the authors in a manner that 
makes clear that the materials are being reused under permission of a Creative Commons License. Views, opinions and conclusions expressed in this 
research article are views, opinions and conclusions of the author(s). Open Access Publishing Group and European Journal of Education Studies shall 
not be responsible or answerable for any loss, damage or liability caused in relation to/arising out of conflicts of interest, copyright violations and 
inappropriate or inaccurate use of any kind content related or integrated into the research work. All the published works are meeting the Open Access 
Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes 
under a Creative Commons Attribution 4.0 International License (CC BY 4.0). 


European Journal of Education Studies - Volume 3 | Issue 10 | 2017 292 


